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Abstract 

We prove a converse Lyapunov theorem for almost sure stabilizability and almost sure 
asymptotic stabilizability of controlled diffusions: given a stochastic system a.s. stochastic 
open loop stabilizable at the origin, we construct a lower semicontinuous positive definite 
function whose level sets form a local basis of viable neighborhoods of the equilibrium. This 
result provides, with the direct Lyapunov theorems proved in a companion paper, a complete 
Lyapunov-like characterization of the a.s. stabilizability. 

Key words. Degenerate diffusion, almost sure stability, stabilizability, asymptotic stability, 
stochastic control, control Lyapunov function, viscosity solutions, Hamilton- Jacobi-Bellman 
inequalities, viability. 

1 Introduction 

In this paper we provide a Lyapunov characterization of almost sure stochastic open loop stability 
at an equilibrium of controlled diffusion processes in M N 

(CSDE) ( dXt = ^ Xtl at ^ dt + a ( X t' a t)dB t , a t 6 A, t > 0, 

This notion has been introduced in a companion paper 6 by Bardi and the author (see also 0): 
we say that (CSDE) is a.s. (open loop) stabilizable if for any 77 > there exists S > such 
that, for any x with \x\ < S, there exists a such that the corresponding process satisfies \X t \ < 77 
for all £ > almost surely. If, in addition, the trajectory is asymptotically approaching a.s. the 
equilibrium, we say the system is a.s. (open loop) asymptotically stabilizable. The definitions imply 
in particular that these properties are never verified by nondegenerate processes. This stochastic 
stability describes a behaviour very similar to a stable deterministic system and is stronger than 
pathwise stability and stability in probability (see ^|E3I^)- We characterize it by means of 
appropriate control Lyapunov functions. These functions have been introduced in [5] and are 
lower semicontinuous (LSC), continuous at the equilibrium, positive definite, proper. Moreover 
they satisfy the following infinitesimal decrease condition: 

sup {-DV(x) -f(x,a) -trace [a(x,a)D 2 V(x)]} > l(x), (1) 

cr(x,a) T DV(x)=0 

where a :— aa T /2, / = for Lyapunov functions and positive definite for strict Lyapunov functions. 
This is not a standard Hamilton- Jacobi-Bcllman inequality, because the constraint on the controls 
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control theory". 
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depends on V: we are allowing diffusion only in the directions tangential to the sublevel sets of V. 
If we eliminate this constraint, the differential inequality which is left is the infinitesimal decrease 
condition on Lyapunov functions for the stability in probability (see J2]jE2)- We prove that V 
satisfies if and only if it satisfies the following monotonicity condition: 

\/x3a : supess sup ( V(X t ) + [ l(X s )ds] < V(x) (2) 
t>o wen \ Jo J 

where the essential supremum is intended with respect to the probability measure . This means 
in particular that the process V(X t ) is a positive supermaxingale according to the definition given 
in |10| : this is the natural counterpart of the requirement on the process V(X t ) to be a positive 
supermartingale in the context of stability in probability. The monotonicity condition says that 
the sublevel sets :— {x V(x) < /i} are viable (or weakly invariant) with respect to (CSDE) in 
the sense that Vie 3a such that X t £ forever almost surely. One of the main tool used 
in this paper is the geometric Nagumo-type characterization of viability proved recently by Bardi 
and Jensen in (see also [2] and the references therein for earlier related results). 

In 0, Bardi and the author show that the existence of a Lyapunov function (respectively, 
of a strict Lyapunov function) implies the a.s. stabilizability (respectively, the a.s. asymptotic 
stabilizability) of the system to the equilibrium. As a simple example of application of this theory, 
we consider a radial function V(x) — v(\x\), for some real smooth function v with v'{r) > for 
r > 0. The system (CSDE) admits V as Lyapunov function if 

Va; 3a : a(x, a) ■ x = f(x, a) ■ x + trace a(x, a) < 0. 

Therefore the following conditions are sufficient for the a.s. stabilizability: the radial component 
of the diffusion is null and its rotational component, which still plays a destabilizing role since 
trace a(x, a) > 0, must be compensated by a negative radial component of /. 

In this paper we prove that the existence of a Lyapunov function is also a necessary condition 
for a.s. stabilizability (and also for a.s. Lagrange stabilizability in the global case). A Lyapunov 
function for the system can be defined as 

V(x) := 'mi{r | 357 admissible control such that \X t \ < r almost surely Vt > 0}, 

or equivalently as 

V{x) := inf supess sup \X^\. 

We prove that this function is LSC, continuous at the origin, positive definite, proper, and satisfies 
the infinitesimal decrease condition with I = 0. For an a.s. asymptotic stabilizable systems in 
a bounded set O, we build a positive definite Lipschitz continuous function /, related to the rate 
of decrease of the stable trajectories to the equilibrium, by the formula 

p + OO 

V{x):= inf ess sup / l(X?)dt. 
aeA x ue Q Jo 

We show that V is finite, LSC, continuous at the origin, positive definite, proper, and satisfies the 
infinitesimal decrease condition 

In both cases the Lyapunov functions are worst- case value function of an appropriate stochastic 
optimal control problem: we minimize the worst possible cost over all possible paths. This is quite 
natural since these functions characterize a very strong stability notion. The link between worst- 
case value functions and viscosity solutions to geometric second order partial differential equations 
has been recently treated by Soner and Touzi in [21] (see also JHI)- They considered stochastic 
target problems where the controller tries to steer almost surely a controlled process into a given 
target by judicial choices of controls. The interest in this kind of stochastic control problems in 
the almost sure setting comes from the relationship with mean curvature type geometric flows and 
from the applications to the super-replication problems in financial mathematics. Moreover the 
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a.s. stability of control systems affected by disturbances modelled as M-dimensional white noise 
is related to the so-called worst-case stability (or robust stability) of deterministic control systems 
affected by disturbances modelled as (deterministic) L°° functions with values in M M (see |T5])- 
The Lyapunov characterization of these two stability properties seems to be an useful tool to give 
a precise proof of this relationship (see J2]), while a direct proof based on the estimates among 
the trajectories of the two systems should be rather hard. An approach of this type has been used 
recently by Da Prato and Frankowska in 14 to prove the equivalence between the invariance with 
respect to a controlled stochastic system and the invariance with respect to a deterministic system 
with two (non competitive) controls. 

We conclude with some additional references on converse Lyapunov theorems. For controlled 
deterministic systems, there are theorems characterizing the stochastic open loop stabilizability 
by means of LSC appropriate Lyapunov functions (see [H]). Soravia in |28j showed that the 
stability at an equilibrium is equivalent to the continuity at such point of the value function 
V{x) = inf ae ^ x sup t>0 U(X t ) where the level sets of U form a local basis of neighborhoods of the 
equilibrium. For asymptotically controllable systems, Sontag and Sussmann ([5B], [2?|) provided 
a characterization of asymptotic controllability by means of continuous Lyapunov functions such 
as V(x) — inf a f^~°° l(X t )dt where I is an appropriate positive definite function. Recently Rifford 
(ESj) proved a converse Lyapunov theorem in the framework of Lipschitz continuous functions 
which are semiconcave outside the equilibrium. In the stochastic setting, Has'minskii (H2|, [T§|) 
obtained a converse theorem for stability in probability of uncontrolled diffusion processes, strictly 
nondegenerate outside the equilibrium, by means of C 2 Lyapunov functions, using the Maximum 
Principle and the properties of solutions of uniformly elliptic equation. Kushner proved in |2()j 
a characterization of asymptotic uniform stochastic stability by means of continuous Lyapunov 
functions (here, however, the infinitesimal decrease condition is given in terms of the weak gener- 
ator of the process). In the forthcoming paper JT] (see also J2|) the author extends the direct 
Lyapunov method by Has'minskii and Kushner to the study of stochastic open loop stabilizability 
in probability in terms of merely semicontinuous Lyapunov functions which satisfy in the viscosity 
sense an appropriate infinitesimal decrease condition and provides also in this setting converse 
Lyapunov theorems. 

The paper is organized as follows. In Section 2 we give the definition of stochastic open loop 
a.s. stabilizability, in Section 3 we introduce the appropriate concept of Lyapunov function for the 
study of such stability. Section 4 is devoted to the viability properties of sublevel sets of Lyapunov 
functions. Section 5 contains the main results: the converse Lyapunov theorems. Finally in Section 
6 we give the extension to general equilibrium sets. 



2 Almost sure Lyapunov stabilizability 

We consider a controlled Ito stochastic differential equation: 

(CSDE) ( dXt = f( Xu a ^ dt + a ( Xt > a t) dB ti t > °> 
\ Xq = x. 

where B t is an M-dimensional Brownian motion. Throughout the paper we assume that /, a are 
continuous functions defined in M N x A, where A is a compact metric space, which take values, 
respectively, in M N and in the space of N x M matrices, and satisfying for all x, y € M N and all 
a e A 

\f(x,a) - f(y,a)\ + \\o-(x,a) - <r(y,a)\\ < C\x - y\, (3) 
We define a(x, a) :— \<j{x 1 a)o~(x, a) T and assume 

{(a(x, a), f(x, a)) : a G A} is convex for all x £ IR N . (4) 

The class of admissible controls is the class of strict controls, as defined in ^] Definition 2.2]: 
they are A valued, progressively measurable processes at such that there exists a solution Xf to 
(CSDE). A x denotes the class of admissible control for a given initial datum x, with a. its generic 



3 



element (although it is not a standard function 1R — > A), and with X. the corresponding solution 
of (CSDE). We recall also a theorem on the existence of optimal control for stochastic control 
problems. 

Theorem 1 (Theorem 4.7 and Corollary 4.8 [18j). Under the convexity assumption J^j), for 
every initial data x G M N there exists an admissible control realizing the minimum in the control 
problem inf a E J(x, a) where the cost functional J(x, a) satisfies standard regularity assumptions. 

We state now the definition of almost sure stochastic open loop stabilizability, which has been 
introduced and studied in the paper |H] (see also [H] for the uncontrolled case). We introduce the 
classes of comparison functions. 

Definition 2 (comparison functions). K. denotes the class of real continuous functions 7 strictly 
increasing and such that 7(0) = 0; /Coo contains the functions 7 G JC such that lim r ^ +00 7(7*) = 
+00. Finally JCC denotes the class of continuous functions (3 : M x 1R — > 1R which are strictly 
increasing in the first variable, strictly decreasing in the second variable, which satisfy (3(0, t) = 
for t > 0, lim t ^ +oc [3(r, t) = for r > 0. 

Definition 3 (a.s. stabilizability). The system (CSDE) is almost surely (open-loop Lyapunov) 
stabilizable at the origin if there exists 7 G /C and 6 > such that for any starting point x with 

\x\ < S 

3a.eA x : |X t |<7(|a;|) Vf > a.s. (5) 

7/7 can be chosen in ICoo and the estimate |3J) holds in the whole space Ft N , the system is also 
almost surely (open-loop) Lagrange stabilizable, or it has the property of uniform boundedncss of 
trajectories. 

Remark 4. We could define the a.s. stabilizability equivalently as follows: 

the system is a.s. (open loop) stabilizable at the origin if for every e > there exists 5 > such 
that for \x\ < 5 there exists an admissible control function a. G A x whose corresponding trajectory 
X. verifies \X t \ < £ for all t > almost surely. 

Remark 5. A necessary condition for the a.s. stabilizability at the origin is that the origin is a 
controlled equilibrium of (CSDE), i.e., 

3a e A : f(0,a) = 0, a(0,a) = 0. (6) 

Definition 6 (a.s. asymptotic stabilizability). The system (CSDE) is almost surely (open 
loop) locally asymptotically stabilizable (or a. s. locally asymptotically controllable^ at the origin 
if there is (3 £ ICC and R > such that for any starting point x with \x\ < R there exists a. G A x 

|X t |</3(|a:|,t) Vi>0a.s. (7) 

3 Lyapunov functions for a.s. stabilizability 

In this section we introduce the appropriate concept of Lyapunov function for the study of the 
almost sure stochastic stability. 

We recall the definition of the second order semijet (see ^Sj) of a LSC function V at x J 2 '~V(x) := 
{(p,Y) G R N xS(N) such that for y -> x, V(y) > V(x)+p-(y-x) + ±(y-x)-Y(y-x)+o(\y-x\ 2 )}. 

Definition 7 (control Lyapunov function). Let O C 1R N be an open set containing the origin. 
A function V : O — > [0, +00) is a control Lyapunov function for the a.s. stabilizability of (CSDE) 
if 

(i) V is lower semicontinuous and continuous at 0; 

(ii) V is positive definite, i.e., V(0) = and V(x) > for all i/O; 

(Hi) V is proper, i.e., the sublevel sets {x\V(x) < /i} are bounded\ffi G [0, 00); 
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(iv) V is a viscosity supersolution in 0\ {0} of the equation: 



sup {-DV{x) ■ f(x, a) - trace[a(x, a)D 2 V(x)}} > 0, 



(8) 



<r(x,a)-DV(x)=0 



in the following sense: for all x G \ {0} and (jp, Y) G J 2, V(x) there exists a G A: 



a(x, a) T p = 



and 



p ■ f(x, a) — trace [a(x, a)Y] > 0. 



If there exists a positive definite, Lipschitz continuous I : O — > 1R such that 



sup {— DV(x) ■ f(x, a) — trace[a(x, a)D 2 V(x)\ } > l(x) 



(9) 



<r(x,a)-DV(x)=0 



then V is a strict control Lyapunov function for the a.s. stabilizability of (CSDE). 

The inequality is not the standard Hamilton-Jacobi-Bellman inequality arising in stochastic 
optimal control. We have an implicit constraint on the controls, o~(x, a)-DV(x) = 0, i.e. depending 
on the generalized subgradients of the solution: we are allowing only controls which render the 
diffusion matrix tangential in some generalized sense to the sublevel sets of V . This implies that 
the diffusion has to degenerate in a large set, for some control. 

Because of this constraint, though, the nonlinearity 



is geometric in the sense that it satisfies the following rescaling property F(x, Xp, XX + fip®p) = 
XF(x,p, X) for every A > and fj, G M, where p ®p is the N x N matrix whose (i, j) entry is PiPj ■ 
This permits to prove the following lemma on the change of unknown (for the proof see [121^. 

Lemma 8. Assume that v is a LSC viscosity supersolution of equation 0) in an open set O. 
Let (j) be a twice continuously differentiable strictly increasing real map. Then w = <f> o v is still a 
viscosity supersolution of equation 0) in O. 

4 Viability properties of Lyapunov functions 

We study now a viability property of the sublevel sets of viscosity supersolution of the nonstandard 
Hamilton-Jacobi-Bellman inequality JSJ). We recall the definition of almost sure viability (named 
also controlled invariance and weak invariance) of an arbitrary closed set for a controlled diffusion 
process. 

Definition 9 (viable set). A closed set K C M N is viable or controlled invariant or weakly 
invariant for the stochastic system (CSDE) if for all initial points x G K there exists an admissible 
control a. G A x such that the corresponding trajectory X, satisfies X t G K for all t > almost 
surely. 

This property was studied by Aubin and Da Prato j2j and, more recently, by Bardi and Jensen 
[Hj. The main result of 0] is the equivalence between the viability of a closed set K and a Nagumo- 
type geometric condition. This geometric condition is given in terms of the second order normal 
cone to a closed set K C M N , first introduced in [5], 



where S(N) is the set of symmetric N x TV matrices. Note that, if dK is a smooth surface in a 
neighborhood of x, p/\p\ is the interior normal and Y is related to the second fundamental form 
of dK at x, see |S]. 



F(x,p, X) — sup {— p ■ f(x, a) — trace[a(x, a)X] \ a G A o~(x, a) ■ p = 0} 



Ml 



\{x) := {(p,Y) E R N x S(N) : for y -> x, y G K, 

P ■ (y - x) + - (y - x) ■ Y(y - x) > o(\y - x\ 2 ) } 
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Theorem 10 (Viability theorem [9 ). Assume conditions 0) and ^j). Then K is viable for 
(CSDE) if and only if 

\/x G dK, V(p, Y) S Nk[x), 3aeA: f(x, a)-p + trace [a(x, a)Y] > 0. 

Moreover, for the same a we have that o~(x, a) ■ p = 0. 

Using this result we obtain the following characterization, which is a variant of a result con- 
tained in |H] (see also 0). 

Lemma 11. Assume conditions @) and ^j). Consider an open set O C M N and a LSC function 

V : O — > M. If V is a viscosity supersolution of 

sup { - DV ■ f(x, a) - trace[a(x, a)D 2 V] } > (10) 

a(x,a)-DV=0 

in O, then the sublevel sets {V(x) < /i} whose boundary is entirely contained in O are viable 
with respect to ( CSDE). Viceversa, suppose JI is the maximal value for which the sublevel set 
{V(x) < fi} has boundary entirely contained in O. Then the function V(x) :— V(x) A JI is a 
viscosity supersolution of in O. 

Observe that if O = M N , every sublevel set of V is closed (in particular we can take JI — +oo). 

Proof. For every fj, < ~p, :— {x | V(x) < /i}. We define now the LSC function V^(x) := [i 
for x € and +oo elsewhere. From the definitions, it is easy to check that J^^V^ix) = 
—J\fft(x), Vi 6 dK^, so, by the Viability Theorem 1101 is a viscosity supersolution of (|10|l if 
and only if is viable. 

We assume that V is a viscosity supersolution of (|1C)|> . Now for A > fixed, we define the 
nondecreasing continuous real function 

!H, t<fi, 
\ 2 {t-ti)+ti, n<t <^ + {, 

The function ip\ o V is a viscosity supersolution of equation (|10|) in O for every A. To prove this 
fact, we choose a sequence tp n of strictly increasing, smooth real maps that converge uniformly on 
compact sets to Then, for every n, the map ip n o V is a viscosity supersolution of equation 
(110(1 in O by Lemma |H1 This permits to conclude, by the stability of viscosity supersolutions with 
respect to uniform convergence. Next we observe that the net %p\ o V is increasing and converges 
as A — > +oo to V M . Viscosity supersolutions are stable with respect to the pointwise increasing 
convergence (see 0]). Therefore the indicator function of is a viscosity supersolution of 
equation l|10(l and then is viable for n <~p. 

Conversely, we assume now that is viable for every fi <~p. Moreover, we observe that 

V(x) = inf V^(x) AJJ = inf{/i < JJ | V(x) < ^} A JI. 
fj,<tj. 

So, by the stability properties of viscosity supersolutions, if V M solves (|10|) for every /i <~p, then 

V solves (Unj too. □ 

This lemma provides the main tool to prove the direct Lyapunov theorems (see jS], 

Theorem 12 (Direct Lyapunov theorem). Assume ^jj. If the system admits a Lyapunov 
function in an open set O containing the equilibrium then 

(i) the system (CSDE) is almost surely stabilizable at the origin; 

(ii) if in addition, the domain O can be chosen as M N , the system is also a.s. Lagrange stabilizable 
and for all x G M N there exists 5 G A x such that the corresponding trajectory X . satisfies 

\X t \ <7r 1 (72(kD) Vi>0 a.s. (11) 



G 



with 71 , 72 6 /Coo • 

Assume moreover that V is a strict Lyapunov function then 

(i) the system {CSDE) is a.s. locally asymptotically stabilizable at the origin; 

(ii) if, in addition, the domain O of V is all M N , the system is a.s. globally asymptotically 
stabilizable. 



5 Converse Lyapunov theorems for a.s. stabilizability 

In this section we prove the main results in the article: we assume that the system (CSDE) satisfies 
an a.s. stabilizability property and construct an appropriate Lyapunov function. 

Theorem 13 (a.s. stabilizability). Assume ^jj. Then 

(i) if the system (CSDE) is almost surely stabilizable at the origin in the ball Bk, the function 



V{x) = 



inf sup ess sup \X" 



AK 



is a Lyapunov function for the system in Bk; 

(ii) if the system is also a.s. Lagrange stabilizable then the function 

V(x) — inf sup ess sup \X"\ 

is a global Lyapunov function for the system. 

Proof. We start proving (i). We start constructing the larger viable set containing the origin in 
the ball Bk- We construct a nonnegative, uniformly continuous, radial function ck such that 
c k{\s\) = if \s\ < K and < ck (\s\) < \s\ for \s\ > K and for A < we consider the value 
function 



r-\-00 

W K (x) = infE, / c K (\X«\)e- Xs ds. 
Jo 



It is well known ([22 and that Wk is a continuous viscosity supersolution of the Hamilton- 
Jacobi-Bellman equation in JR N 

max {-DW(x) ■ f(x, a) - trace [a(x, a)D 2 W(x)] } + XW(x) > c K (\x\) 

The function Wk is nonnegative in M N and Wk (0) = since the origin is a controlled equilibrium 
as remarked in ©. We consider the propagation set of the minimum value 0: 

Prop(0, W K ) = {x G M N \ W K (x) = 0} = {x \ 3a : X t € ~Bk~ a.s.Wt} C U^. 

This set is clearly closed and it can be proved that it is also viable. This follows immediately from 
Theorem 4.6 in Bardi, Da Lio |7] or can be checked directly. 

The candidate Lyapunov function is defined in the set O = Bk as 

V(t) = I [niaeA - su P*>o ess su P^eo x e PropjO, W K ) 

1 ' \ K xeB K \ Prop(0, W K ). 

First of all we observe that the function V is well defined: by definition of a.s. stabilizability, there 
exists a function 7 £ JC such that V(x) < 7(|x|). From this we get also that V is continuous at 
the origin. We observe also that Prop(0, Wk) 12 ^(0,7 _1 (^)). Moreover V is positive definite. 
Indeed if V(x) = then for every e > there exists a £ such that the corresponding trajectory 
satisfies \X t \ < e for all t > almost surely: so inf Q Eo f Q °° \X t \e~ xt dt = for any fixed 
A > 0. By Theorem ^ the inf is attained, and the minimizing control produces a trajectory 
satisfying a.s. \X t \ — for all t > 0. 
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To prove the semicontinuity and the differential inequality, we provide another characterization 
of V. We prove that it coincides with 

w(x) := inf{r | 3a G A x \X t \ < r Vt > a.s.} A K. (12) 

First of all with the usual argument based on Theorem ^ we get that this infimum is actually a 
minimum. Then we fix K > k > w(x): by definition there exists a G A x such that \X t \ < k — e 
for all t > a.s. This means that sup t>0 esssup wen \Xt\ < k from which we deduce w(x) > V(x). 
The converse is similar. 

Therefore, for every < fj, < K, the sublevel set {x £ O \ V(x) < /i} coincides with {x G 
M N | 3a G A x \X t \ < /i Vi > a.s.}; in particular this gives that the function V is proper. As at 
the beginning of the proof, we can characterize these sets as the propagation sets of the minima 
of viscosity supersolutions of suitable Hamilton- J acobi- Bellman equations. Then, by the results 
in |2|, these sets are closed in M N and viable with respect to (CSDE). So the function V is LSC. 
Moreover, by Lemma ITT1 it satisfies the infinitesimal decrease condition JSJ. 

To prove (ii), we observe that for every K > we can repeat the previous construction, since 
the estimate JHJ holds in the whole space with 7 G /Coo. So we get an increasing sequence of 
Lyapunov functions Vk'- for every K > we construct as before the function Vk in the ball Bk 
and extend it to the whole space in the obvious way. Hence the global Lyapunov function for the 
system is 

V(x) = lim Vk{x) — sup Vk(x) — inf supesssup \X"\. 

It is immediate to check that it satisfies the condition (i),(ii),(iii) in the Definitional Moreover the 
proof of the fact that V satisfies the differential condition (iv) relics on standard stability properties 
of viscosity supersolutions with respect to the pointwise increasing convergence (0]). □ 

Now we prove the converse Lyapunov theorem for asymptotic stability in a bounded set. 

Theorem 14 (a.s. asymptotic stabilizability). Assume ^p. If the system (CSDE) 
is a.s. asymptotically stabilizable at the origin in the ball Bk, then there exist an open set O 
containing the origin and a Lipschitz continuous positive definite function I : O —> 1R such that 

r+oc 

V{x)= inf ess sup / l{X?)dt 
is a Lyapunov function for the system in O. 

Proof. The estimate (J7J permits to construct a positive definite Lipschitz continuous function 
I : M N — > 1R as follows. t x (U) denotes the random time spent in the set U by the trajectory 
X. Using the properties of the function (3 we get that, for every r < K and x G Bk, the time 
t x (M N \ B r )(uj) spent by the trajectory X outside B r is almost surely bounded and moreover it 
satisfies 

sup esssupt^iR \B r )(uj) < +00. 

We consider now a decreasing sequence of positive numbers such that ro < K and lim^ +oc = 
0. We define 

T,i= sup ess sup t x (M N \ B Ti ){w). 

The sequence of positive numbers T% is increasing as i — > +00: we can choose a decreasing sequence 
of positive numbers U such that X^S? = ^ < +°°- The function / : 1R N — > M is therefore 
defined as a radial Lipschitz continuous, positive definite, nondecreasing function which satisfies 
1(0) — 0, l(\x\) = /j+i for \x\ = Ti and l(\x\) = l\ for every |a;| > ro. 
The candidate Lyapunov function is 

p + CC 

V(x) = inf ess sup / l(\X"\)dt for x G Bk- 
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The rest of the proof will be devoted to show that this function satisfies the properties of Definition 
and then is a strict Lyapunov function. First of all V is well defined: 

V(x) = inf ess sup / l(\X?\)dt < ess sup / l(\~X t \)dt < = M. 



i=0 



By definition V(x) > for every x and V(0) — 0. We assume now that for some x ^ V(x) = 0: 



this means that for every e > there exists a e £ A x such that J +O ° Z(|X t Qe |)c?i < e almost surely. 
Then info, ~E X / + °° l(\X"\)dt = 0: from this, by the usual argument based on Theorem we 
deduce that x = 0. We show now that the function V is continuous at the origin. Recalling the 
definition of t x (M N \ B r ) and the a.s. asymptotic stabilizability, we get that t x (M N \ B r ) = 
almost surely for initial data x such that /?(|x|,0) < r. Therefore, by the continuity at r = of 
the function /3(r, 0), for every e > there exists 5 > such that, for \x\ < 5, /3(|x|,0) < e and 
then t x (M N \ Bf.) — almost surely for every k > e. So we get V(x) < 53i=?(e) ^i^i where i(e) is 
the minimum index for which < e. Since the sum l{Ti converges and — > as i — > +oo, 
for every 9 > 0, we can choose e > such that J2t=*i(e) < ^ : this gives the continuity at the 
equilibrium. To conclude the proof we have to provide another equivalent definition of V. We 
consider the new system in M N+1 

(CSDE2){ d ( X t,Yt) =7(Xt,Y t ,a t )dt + a(X t ,Y t ,a t )d(B t ,0), t > 0, 
\ (X ,Y ) = (x,y). 

where f{x,y,a) = {f{x, a), l{x)) a.nd'a{x,y,a) — (a(x, a),0). It satisfies conditions © and 
and has (0, 0) as a controlled equilibrium. We introduce now the following function 

W(x, y) = inf{r 3a e A x \Y t a \ < r Vi > a.s.} = 



if{r | 3a € A 



-Hoc 



y+ l(\X?\)dt 



< r a.s.}. 



Using Theorem^ it can be proved easily that this infimum is actually a minimum. So the sublevel 
sets of W are 



{(x,y) | W(x,y) < n} = \ (x,y) \ 3a e A 



+ 00 



y+ / l{\X?\)dt 



< /i a.s. 



Repeating the argument in the proof of Theorem ^3 we get that these sets are closed and viable 
with respect to (CSDE2). So the function W(x, y) is LSC and satisfies, by Lemma ITT1 in viscosity 
sense 

sup {-D X W ■ f(x,a) -trace[a{x,a)Dl x W]} -l(\x\)D y W > 0. (13) 

er(>,a)-,D x W=0 

Now we show that the candidate Lyapunov function V coincides with the function W on the set 
Br x {y = 0}. Assume that V(x) < r. For every e > 0, there exists a e € A x such that almost 
surely l(\X?' \)dt <r + e. Therefore W(x,0) < r + e and so we conclude by the arbitrariness 
of e. The proof of the opposite inequality V(x) < W(x, 0) is similar. From this characterization of 
the function V we deduce immediately that it is LSC and bounded in Br. It remains to check the 
differential condition JjJJ. We fix x / in Bk and consider (p, Y) G J 2 '~V(x): by definition, for 
every x' -> x, we get W(x',0) > W(x, 0) +p- (x' -x) + l/2(x' -x)Y(x' -x) +o(\x - x'\). Using the 
definition, it is immediate to check W(x', 0) < W(x', y) — y. This implies that if (p, Y) G J 2 '~V{x) 
then (p,Y, 1) G J 2 ' l ~W{x,Q). So the differential condition Q comes from (j 1 r?|l . □ 

6 Extensions 

We can extend the results to the characterization of stabilizability of a closed set M C M . We 
denote with d(x, M) the distance between x G M N and M. 
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Definition 15 (a.s. stabilizability at M). The system (CSDE) is almost surely (open loop) 
stabilizable at M if there exists 7 G /C such that, for every x in a neighborhood of M , there is an 
admissible control function a. G A x whose trajectory X. verifies 

d(X t ,M) < j(d(x, M)) Vf > almost surely. 

Remark 16. From this definition, using Theorem^ we can deduce that if M is a.s. stabilizable, 
then it is viable for {CSDE). 

We adapt the definition of control Lyapunov function to the case the equilibrium is a set M: 

Definition 17 (control Lyapunov functions at M). Let O be an open neighborhood of the 
closed set M . A function V : O — > [0, +00) is a control Lyapunov function at M for (CSDE) if 

(i) V is lower semicontinuous; 

(ii) there exist 71,72 6 IC such that j2(d(x,M)) < V(x) < r )\(d(x, M)) for all x 6 O; 

(Hi) for all x G \ M and (p, Y) G J 2 '~V(x) there exists a £ A such that condition 0) holds. 

Theorem 18. Assume ^j). Then the system (CSDE) is almost surely stabilizable at M if 
and only if there exists a control Lyapunov function V at M. 
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